Lip-sync correcting device and lip-sync correcting method

ABSTRACT

A video signal and an audio signal are transmitted from a source device ( 400 ) to a sink device ( 410 ) in conformity with respective predetermined interfaces through respective transmission channels. The respective interfaces include bidirectional interface such as an HDMI and an IEEE1394. A controller ( 420 ) acquires delay information (TLv) representing the total delay time of the devices connected to the transmission channels conforming to the bidirectional interface and corrects the difference in reproduction time between the video and audio signals by using the acquired delay information (TLv).

TECHNICAL FIELD

The invention relates to an apparatus and a method of correcting timedifference of a video signal and an audio signal, that is, deviation inLip-sync.

BACKGROUND ART

Lip-sync is a synchronizing technology of a video signal and an audiosignal in reproduction, and various methods have been proposed so far torealize Lip-sync. FIG. 24 shows an example of a conventional video-audiosynchronizing system. In a source device 210, a same time stamp (timecode) is multiplexed on video signal and audio signal. The video signaland audio signal on which the time stamp is multiplexed pass throughvarious paths such as the repeaters 214 and 216, and both are finallysupplied to a sink device 220. In the sink device 220, the time stamp ofvideo signal and the time stamp of audio signal are retrieved, and thedifference is detected by a time code comparator 222. On the basis ofthe detected difference, a video delay device 224 and an audio delaydevice 226 located at the final points of the respective paths arecontrolled. For example, when the video signal is later than the audiosignal, an extra delay is given to the audio signal, and the total delaytime of both signals is controlled to be identical.

Patent document 1 discloses a technology for automatically correctingthe time deviation of audio signal, by comparing the time stamps ofvideo signal and the audio signal, and delaying either the video signalor audio signal, so that the both may be matched.

** Patent document 1: JP-A-2003-259314

DISCLOSURE OF THE INVENTION Problems to be solved by the Invention

In a conventional system, when the time stamps of video signal and audiosignal can be compared and detected at the same time, that is, when bothtime stamps are supplied in the same device at the same time, thedeviation can be detected and Lip-sync can be corrected. However, in atopology composed of separate transmission paths of video signal andaudio signal, since only one time stamp can be recognized, the timedifference of video signal and audio signal cannot be detected and theLip-sync cannot be corrected.

The invention is directed to solve the above problem, and it is hence anobject thereof to present a Lip-sync correcting apparatus capable ofcorrecting the Lip-sync securely even in a topology of separate passesof the video signal and audio signal.

In a first aspect of the invention, provided is a Lip-sync correctionsystem for correcting difference in reproduction time between a videosignal and an audio signal. The Lip-sync correction system transmits avideo signal and an audio signal from a source device to a sink devicethrough the respective transmission paths in conformity with a specifiedinterface, individually. The specified interface includes abidirectional interface. The Lip-sync correction system includes acontroller for acquiring delay information indicating the total delaytime of devices on the transmission paths conforming to thebidirectional interface, and correcting the difference in reproductiontime by using the acquired delay information.

According to the configuration, the controller acquires delayinformation of devices through transmission paths of the bidirectionalinterface, and adjusts the difference in reproduction time between avideo signal and an audio signal by using the acquired information.According to the method, the controller can adjust the reproduction timedifference between a video signal and an audio signal on onetransmission path on the basis of the delay time information obtainedfrom the other transmission path. Thus, the Lip-sync can be correctedsecurely even if a video signal and an audio signal are transmitted indifferent transmission paths.

In the Lip-sync correction system, the video signal may be transmittedin a bidirectional interface, while the audio signal may be transmittedin a one-way interface. The delay information of a video signal for thedevice on the transmission path can be acquired from the downstream ofthe transmission path of a video signal by transmitting the video signalin the bidirectional interface. The acquired delay information can betransmitted from the upstream of the transmission path of an audiosignal. As a result, Lip-sync can be controlled in the device on thetransmission path of an audio signal.

The controller may apply an extra delay to the device on thetransmission path of an audio signal in order to correct difference inreproduction time. Generally, the video signal is later than the audiosignal, and therefore the Lip-sync can be corrected securely by applyingan extra delay to the device on the transmission path of an audiosignal.

The controller may also apply an extra delay to the device at thelowermost downstream on the transmission path of an audio signal.Adjustment with the extra delay time in the device at the lowermostdownstream allows Lip-sync correction to be realized more securely andmore precisely.

The delay information may be transmitted as additional information of anaudio signal. Hence, even if the transmission path of an audio signal isa one-way interface, the delay information of a video signal can beacquired through the transmission path of an audio signal.

In a second aspect of the invention, provided is a video reproducingapparatus for generating and outputting a video signal and an audiosignal. The video reproducing apparatus includes a video interface unitwhich is connected to a video signal transmission path and is operableto transmit a video signal in conformity with a bidirectional interface,an audio interface unit which is connected to an audio signaltransmission path, and is operable to transmit an audio signal, and acontroller for acquiring delay information indicating total delay timeof devices connected to the video signal transmission path, andtransmitting the acquired delay information to a device connected to theaudio signal transmission path.

In a third aspect of the invention, provided is an audio output devicefor receiving an audio signal, and outputting audio on the basis of thereceived audio signal. The audio output device includes an interfaceunit operable to receive delay information about a video signal anddelay information about an audio signal, together with the audio signal,and a controller operable to adjust output time of an audio signal onthe basis of the received delay information.

In a fourth aspect of the invention, provided is a video display devicefor receiving a video signal and presenting video on the basis of thereceived video signal. The video display device includes an interfaceunit operable to receive a command for requesting transmission of delaytime, and a controller operable to transmit the delay time of a videosignal for the video display device through the interface when receivingthe command.

In a fifth aspect of the invention, provided is a Lip-sync correctingmethod for correcting difference in reproduction time between a videosignal and an audio signal. The Lip-sync correcting method transmits avideo signal and an audio signal from a source device to a sink devicethrough the respective transmission paths in conformity with therespective specified interfaces, individually. The specified interfaceincludes a bidirectional interface. The Lip-sync correcting methodacquires delay information indicating the total delay time of devices onthe transmission paths conforming to the bidirectional interface, andcorrects the difference in reproduction time between a video signal andan audio signal by using the acquired delay information.

EFFECT OF THE INVENTION

According to the invention, the controller acquires delay information ofdevices through the transmission paths of a bidirectional interface, andcorrects the difference in reproduction time between a video signal andan audio signal by using the acquired information. Thus, the controllercan adjust the reproduction time difference between a video signal andan audio signal on one transmission path on the basis of delay timeinformation obtained from the other transmission path. Hence, theLip-sync can be corrected securely even if the transmission paths of avideo signal and an audio signal are different.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory diagram of basic concept of the invention.

FIG. 2 is a system block diagram of a system configuration in the firstembodiment of the invention.

FIG. 3 is a block diagram of hardware of a source device (DVD player).

FIG. 4 is a block diagram of hardware of a sink device (video displaydevice).

FIG. 5 is a block diagram of hardware of a sink device (audio outputdevice).

FIG. 6 is a block diagram of hardware of a repeater.

FIG. 7 is a block diagram of HDMI.

FIG. 8 is a flowchart of general operation of a system in the firstembodiment.

FIG. 9 is a flowchart of acquisition process of total latency of a videosignal TLv by a source device in the first embodiment.

FIG. 10 is a flowchart of a process of a repeater when receiving TLvtransmission command.

FIG. 11 is a flowchart of a process of a sink device when receiving TLvtransmission command.

FIG. 12 is a flowchart of a process for transmission of total latency ofa video signal TLv and cumulative delay time sDa of an audio signal froma source device.

FIG. 13 is a flowchart of transmission process of total latency of videosignal TLv and cumulative delay time of audio signal sDa.

FIG. 14 is a flowchart of a process of adjustment of audio output timein a sink device.

FIGS. 15A to 15C are system block diagrams each indicating a systemconfiguration in the second embodiment of the invention.

FIGS. 16A to 16C are system block diagrams each indicating a systemconfiguration in the third embodiment of the invention.

FIGS. 17A to 17C are system block diagrams each indicating a systemconfiguration in the fourth embodiment of the invention.

FIGS. 18A to 18C are system block diagrams each indicating a systemconfiguration in the fifth embodiment of the invention.

FIGS. 19A and 19B are system block diagrams each indicating a systemconfiguration in the sixth embodiment of the invention.

FIGS. 20A to 20C are system block diagrams each indicating a systemconfiguration in the seventh embodiment of the invention.

FIGS. 21A to 21C are system block diagrams each indicating a systemconfiguration in the eighth embodiment of the invention.

FIG. 22 is a schematic diagram for explaining a transmission method ofadditional information to be added to an audio signal.

FIG. 23 is a diagram of format of additional information, such aslatency information, additionally specified according to “General userdata format”.

FIG. 24 is a system block diagram of connection configuration of aLip-sync correcting apparatus in prior art.

REFERENCE SIGNS

-   100 Source-   101 DVD media-   102 Audio variable delay unit-   110 Repeater-   112 Re-encoder-   120 Repeater-   122 Video signal processing unit-   130 Repeater-   132 Signal processing unit-   140 Sink device-   142 Video signal processing unit-   143 LCD (liquid crystal display device)-   150 Sink device-   151 Audio variable delay unit-   152 Decoder-   153 Speaker

BEST MODE FOR CARRYING OUT THE INVENTION

Referring now to the accompanying drawings, preferred embodiments of theinvention are described below. In the drawings, the same or similarconstituent elements are identified with the same reference numerals.

Basic Concept

First of all, a basic concept of the invention about Lip-sync correctionis explained by referring to FIG. 1.

The invention is intended to correct deviation in Lip-sync, in an audioand video reproduction system which includes a source device 400 forgenerating an audio signal and a video signal and a sink device 410 forreproducing and outputting an audio signal and a video signal, with thesource device 400 connected to the sink device 410 directly or by way ofa repeater. The invention also includes a controller 420 achieving afunction for correcting deviation of Lip-sync by applying an extradelay.

In the present invention, a video signal is transmitted through a videotransmission path using a bidirectional data interface such as IEEE1394or HDMI. The source device 400 inquires total latency of a video signal(sum of latency of devices on the video transmission path) through thevideo transmission path, with command sequence in IEEE1394, or with EDID(Extended Display Identification Data) in HDMI (High-DefinitionMultimedia Interface). The source device 400 acquires information oftotal latency through an up-line and transmits the information of totallatency to other devices on an audio transmission path. The otherdevices on the audio transmission path transmit audio latencyinformation accumulated in each device together with the video signaltotal latency information. The final device on the audio transmissionpath sends the sum of audio latency as sDa together with video signaltotal latency (TLv) through the audio transmission path.

The controller 420 detects sDa and TLv only from the final device. IfTLv>sDa, the controller 420 adds an extra delay (TLv-sDa) to the finaldevice on the audio transmission path.

Thus, according to the present invention, total latency information of avideo signal (delay information) is acquired preliminarily, and theacquired information is transmitted to the device on the audiotransmission path through the audio transmission path. On the basis ofthe finally obtained latency information of a video signal and an audiosignal, the output time of an audio signal in the final device isadjusted, thus correcting the deviation of a video signal and an audiosignal, that is, deviation in Lip-sync.

The function of the controller 420 typically is implemented in, forexample, a sink device. However it may be implemented in a device otherthan the sink device on the audio transmission path (for example, asource device or a repeater) as far as the function of adding extradelay to the final device can be realized on the basis of the finallyobtained total latency information. The extra delay may not be alwaysadded to the final device on the audio transmission path, and it may beadded to any one of devices on the audio transmission path. The extradelay may also be dispersed and added to a plurality of devices.

On the basis of the concept described above, some of specificembodiments of the invention are explained below.

EMBODIMENT 1 1. System Configuration

FIG. 2 is a block diagram of a video and audio reproduction system towhich the concept of Lip-sync correction of the present invention isapplied.

The video and audio reproduction system includes a source device 100 forgenerating video and audio signals, repeaters 110 and 130 for amplifyingthe audio signal, a repeater 120 for amplifying the video signal, a sinkdevice 140 for displaying video on the basis of the video signal, and asink device 150 for outputting audio on the basis of the audio signal.

1.1 Source Device

The source device is located at the uppermost upstream of the audio orvideo transmission path, and serves as an output source of a video oraudio signal. In the embodiment, the source device 100 is a DVD playeras a reproducing apparatus of a DVD medium 101, and generates andoutputs a video signal 200 and an audio signal 300.

FIG. 3 shows a hardware configuration thereof. The source device 100includes a pickup 11 for reading information from the DVD medium 101 andconverting the information to an electrical signal, a front endprocessor 13 for receiving an output signal from the pickup 11 andgenerating video and audio signals, a system controller 15 forcontrolling the entire operation of the source device 100, a RAM 17serving as a work area, and a ROM 19 for storing specified information.The source device 100 also has an HDMI interface unit 21 for exchanginga video signal and others with an external device, and an IEC interfaceunit 22 for exchanging an audio signal and others. The source device 100realizes the following functions and processes by execution of aspecified program by the system controller 15. As shown in FIG. 2, thesource device 100 has an audio variable delay unit 102, which delays theaudio signal 300 by a specified time (20 ms in this example) and outputsit.

The source device is not limited to a DVD player, but it may be realizedby a reproducing apparatus using other media such as a hard disk player.

1.2 Sink Device

The sink device is located at the lowermost downstream of the audio orvideo transmission path, and outputs a video or audio signal.

In the embodiment, the sink device 140 is a video display device and hashardware configuration as shown in FIG. 4. The sink device 140 includesHDMI interface unit 31 for receiving a digital video signal inconformity with HDMI interface, a video decoder 33 for decoding thereceived digital video signal, a liquid crystal display device (LCD) 143for displaying videos, a driver 37 for generating a video signal fromthe decoded video signal and driving the LCD 143, a controller 39 forcontrolling the entire operation of the sink device 140, a RAM 41serving as a work area of the controller 39, and a ROM 43 for storingspecified information. The sink device 140 realizes the followingfunctions and processes by execution of a specified program on thecontroller 39. The video decoder 33 and driver 37 shown in FIG. 4compose a video signal processing unit 142 shown in FIG. 2. Videolatency Lv of the video signal processing unit 142 is 80 ms. “Latency”is delay time from input to output of a signal in a device. Informationof video latency Lv is stored in the ROM 43 of the sink device 140.

The sink device 150 is an audio output device, and has a hardwareconfiguration as shown in FIG. 5. As shown in FIG. 5, the sink device150 includes an IEC interface unit 51 for receiving a digital audiosignal in conformity with IEEE60958, an audio decoder 152 for decodingthe received digital audio signal, a D/A converter 55 for converting thedecoded signal into an analog signal, an amplifier 57 for amplifying theanalog signal, a speaker 153 for outputting audio according to theoutput from the amplifier 57, a controller 61 for controlling the entireoperation of the sink device 150, a RAM 63 serving as a work area of thecontroller 61, and a ROM 65 for storing specified information. Referringto FIG. 2, the sink device 150 has an audio variable delay unit 151 fordelaying an audio signal by specified time (10 ms in this example).Audio latency La of the decoder 152 is 10 ms. Information of audiolatency La is stored in the ROM 65 of the sink device 150.

The number and type of sink devices are not particularly limited tothose stated above.

1.3 Repeater

The repeater is a device disposed somewhere in the audio or videotransmission path, and is an amplifier, for example. FIG. 6 shows thehardware configuration. The repeater 110 includes an HDMI interface unit71 for receiving a digital video or audio signal in conformity withHDMI, an IEC interface unit 72 for receiving a digital video or audiosignal in conformity with IEEE1394, a controller 73 for processing thereceived digital signal, a RAM 75 serving as a work area of thecontroller 73, and a ROM 79 for storing specified information such asprogram. The controller 73 executes the specified program to realize thefollowing functions and processes.

Referring to FIG. 2, the repeater 110 receives a video signal 200 and anaudio signal 300 from the source device 100, amplifies and outputs themas a video signal 201 and audio signal 301. The repeater 110 includes are-encoder 112, which once decodes the audio signal 300 to read out theinformation, and encodes the read information again. Latency La of thisre-encoder 112 is 10 ms. Information of latency La is stored in the ROM79 of the repeater 110.

The repeater 120 has a video signal processing unit 122 for amplifying avideo signal, and receives a video signal 201, amplifies, and outputs itas a video signal 202. Latency Lv of the video signal processing unit122 is 20 ms. The repeater 130 includes a signal processing unit 132 forprocessing the audio signal 301 as specified, and outputs the processedsignal as an audio signal 302. Latency La of the signal processing unit132 is 50 ms. The repeaters 120 and 130 have the same structure as shownin FIG. 6, and store the respective latencies in the ROMs respectively.

The number and type of repeaters are not particularly limited to thoseabove stated.

1.4 Interface

In the embodiment, HDMI (High-Definition Multimedia Interface) is usedas a transmission interface for a video signal. “HDMI” is a standard ofan interface of input/output of digital audio and video, which isestablished in December in 2002 and mainly directed to electrichousehold appliances and AV devices. In conformity with HDMI, video,audio and control signals can be sent and received together through asingle cable, and optionally a control signal can be transmittedbidirectionally.

In the present invention, with HDMI, the up-line function is realized,that is, a digital video signal is transmitted at high speed from thesource device 100 to the sink device 140. Further, profile informationof the sink device 140 is transmitted sequentially to the repeater 120,repeater 110, and source device 100. This up-line function is calledhereinafter “EDID” (Extended Display Identification Data) line”.

FIG. 7 shows a configuration of HDMI. As shown in the diagram, HDMI hasthree data channels and one clock channel, through which video data,audio data and other data are transmitted. HDMI also has a display datachannel (DDC) for exchanging information on structure or status ofdevices. Further, HDMI has an optional CEC line through which variouscontrol signals can be transmitted bidirectionally among various AVappliances. In the present embodiment, information about latency ofvideo and audio of appliances is transmitted through DDC. Instead ofDDC, the CEC line may be used to transmit information about latency. Asimilar function can be realized by using IEEE1394 instead of HDMI.

Details of HDMI and EDID are disclosed in the following publication, forexample:

“High-Definition Multimedia Interface Specification Version 1.1”,Hitachi Ltd. et al., May 20, 2004, Internet<http://www.hdmi.org/download/HDMI_Specification_(—)1.1.pdf>.

In the present embodiment, IEEE60958 as one-way interface is used forthe audio signal transmission. In the following embodiments, HDMI asbidirectional interface may be also used for the audio signaltransmission.

2. Operation

In the system configuration stated above, the video signal 200 istransmitted from the source device 100 to the sink device 140 by way ofthe repeaters 110 and 120 in conformity with HDMI interface. The audiosignal 300 is transmitted from the source device 100 to the sink device150 by way of the repeaters 110 and 130 in conformity with IEC60958interface.

In the video transmission path, the total video latency TLv from thesource device 100 to the sink device 140 is sum of latency Lv (20 ms) ofa video signal processing unit 122 of the repeater 120 and latency Lv(80 ms) of a video signal processing unit 142 of the sink device 140,resulting in 100 ms.

In the audio transmission path, the sum of the audio latency La ofdevices from the source device 100 to the sink device 150 is the sum oflatency La (10 ms, 50 ms and 10 ms) of the re-encoder 112 of therepeater 110, the repeater 130 and the sink device 150, resulting in 70ms.

That is, the total video latency TLv is 100 ms, and the sum of latencyLa of audio signal is 70 ms (an adjustable delay time of the audiovariable delay unit 151 is not included). Hence, the audio signal isreproduced earlier by 30 ms. Process of this system to solve the timedifference between the video signal and audio signal in reproductionoperation is described below.

2.1 General Flow

FIG. 8 is a flowchart of general operation of the system. First, thesource device at the uppermost upstream acquires the total video latencyTLv of devices on the video transmission path (S11). Details of thisprocess are explained later.

In the audio transmission path, the total video latency TLv, and the sumof delay time and latency of audio signals of devices (referred to as“cumulative delay time” hereinafter) sDa are transmitted from the sourcedevice 100 to the sink device 150 (S12). At that time, the cumulativedelay time sDa of the audio signal is sequentially transmitted to thedevices on the audio transmission path, while the value of latency ofeach device is cumulated and transmitted sequentially to the downstream.

Finally, in the sink device at the lowermost downstream, the audio isdelayed and output on the basis of the value of the cumulative delaytime sDa of the audio signal (S13). Thus, the audio output time isadjusted, and the difference between the video output time and the audiooutput time can be resolved.

2.2 Acquisition of Video Signal Total Latency TLv

Details of acquiring operation of the video signal total latency TLv(step 11 in FIG. 8) is explained.

2.2.1 Acquisition of TLv by Source Device 100

Referring to FIG. 9, acquisition of total latency TLv of the videosignal by the source device 100 is explained. The source device 100sends a TLv send command to devices (repeaters, sink device) at thedownstream of the video transmission path (S41). When the TLv sendcommand is sent from the source device, the command is sequentiallytransmitted to the downstream devices. Then, from the downstream to thesource device 100, information sLv whish is calculated by sequentiallyadding latency Lv of each device is transmitted. Finally, the cumulativevalue of latency Lv of devices (repeaters, sink device) (referred to as“cumulative latency” hereinafter) sLv is transmitted to the sourcedevice 100. Details of this process are described below.

When receiving the cumulative latency sLv from the downstream devices(S42), the source device 100 reads out the own latency Lv from the ROM19 (S43), and adds the read latency Lv to the cumulative latency sLvreceived from the downstream devices (S44). Thus, the total latency TLvsof video signal can be obtained.

2.2.2 Transmission of sLv from Repeater

Referring to FIG. 10, operation of the repeater 120 upon receiving a TLvsend command is explained. When receiving a TLv send command (S31), therepeaters 110 and 120 transfer the TLv command to the downstream devices(repeater, sink device) (S32), and wait for transmission of thecumulative latency sLv from the downstream devices (S33). When receivingthe cumulative latency sLv from the downstream devices, the repeaters110 and 120 read out the own latency Lv from the ROM 79 (S34). Theobtained latency Lv is added to the cumulative latency sLv transmittedfrom the downstream devices (S35), and the newly obtained cumulativelatency sLv is sent to upstream devices (repeater, source device) (S36).

2.2.3 Transmission of sLv from Sink Device

Referring to FIG. 11, operation of the sink device 140 when receiving aTLv send command is explained. The sink device 140 receives a TLv sendcommand (S21), and reads the own latency Lv from the ROM 43 (S22). Theread latency Lv is sent to the upstream devices (repeater, sourcedevice) as cumulative latency sLv (S23).

2.3 Transmission of Video Signal Total Latency TLv and Audio SignalCumulative Delay Time sDa

Transmission operation of the video signal total latency TLv and theaudio signal cumulative delay time sDa (step S12 in FIG. 8) is detailedbelow.

In the audio transmission path, the video signal total latency TLv andthe audio signal cumulative delay time sDa are sequentially sent fromthe source device 100 to the sink device 150. At this time, the latencyof each device on the audio transmission path is sequentially added tothe audio signal cumulative delay time sDa, and finally the total valueof latency La of all devices on the audio transmission path except forthe device at the lowermost downstream (sink device 150) is transmittedto the device at the lowermost downstream (sink device 150).

2.3.1 Transmission of TLv and sDa by Source Device

Referring to FIG. 12, transmission operation of the video signal totallatency TLv and the audio signal cumulative delay time sDa from thesource device 100 is explained.

The source device 100 receives the video signal total latency TLv (S51),reads but own latency of audio signal La from the ROM 19 (S52), andsends the read latency La as audio signal cumulative delay time sDa tothe downstream devices on the audio signal transmission path (S53).

2.3.2 Transmission of TLv and sDa by Repeater

Referring to FIG. 13, transmission of the video signal total latency TLvand the audio signal cumulative delay time sDa is explained.

The repeaters 110 and 130 receive the video signal total latency TLv andthe audio signal cumulative delay time sDa from the upstream device(S61), and read out own audio signal latency La from the ROM 79 (S62).The repeaters 110 and 130 add the read latency La to the receivedcumulative delay time sDa, and send the result to the downstream devices(repeater, sink device) (S63).

2.4 Adjustment of Audio Output Time

Referring to FIG. 14, adjusting operation (step S13 in FIG. 8) of audiooutput time in the sink device 150 is explained.

The sink device 150 receives the video signal total latency TLv and theaudio signal cumulative delay time sDa from the upstream devices (S71),and then determines delay time (AD) for output time adjustment (S72).The delay time (AD) is obtained by subtracting the signal cumulativedelay time sDa from the video signal total latency TLv. Audio ispresented with the output timing delayed by the specified delay time(AD) (S73).

2.5 Specific Examples of Acquisition of Total Latency TLv

Individual processes are described above, and flow of entire system ofthe configuration shown in FIG. 2 is explained below.

The source device 100 issues a TLv send command for acquiring videosignal total latency TLv for devices from the source device 100 to thesink device 140. The TLv send command causes the sink device 140 totransmit “sLv=80 ms” which is a parameter showing latency Lv (80 ms) ofthe sink device 140, to the repeater 120 by way of EDID line. Therepeater 120 transmits “sLv=100 ms” which is the sum of a valueindicated by the received parameter of “sLv=80 ms” and the latency (20ms) of itself, to the upstream repeater 110. The repeater 110 which haveno video signal latency transfers the received value of “sLv=100 ms”directly to the source device 100.

The source device 100 set the received sLv (=100 ms) to the video signaltotal latency TLv (100 ms). The source device 100 multiplexes the videosignal total latency TLv (100 ms), as a fixed parameter “TLv=100 ms”, onthe audio signal 300 to transmit it. At the same time, the source device100 transmits the audio signal cumulative delay time sDa as additionalinformation of the audio signal 300 to the sink device 150. In theexample in FIG. 2, in the source device 100, the delay time 20 ms is setin the audio variable unit 102. Hence, the source device 100 multiplexesthe parameter “sDa=200 ms” indicating delay time 20 ms of the audiovariable delay unit 102 on the audio signal 300 and transmits it.

The repeater 110 receives a parameter “sDa=20 ms” from the source device100, and accumulates the own latency La (10 ms) with the value indicatedby the parameter, and transmits “sDa=30 ms” which is multiplexed on theaudio signal 301. Similarly, the repeater 130 adds the own latency La(50 ms) to the cumulative delay time sDa (30 ms), and outputs “sDa=80ms” which is multiplexed on the audio signal 302.

The sink device 150 reads the cumulative delay time sDa (80 ms), addsthe own latency La (10 ms) to the delay time to obtain 90 ms as audiosignal total delay time before correction. The sink device 150,similarly, reads the fixed parameter “TLv=100 ms” showing that the videosignal total latency TLv is 100 ms. Thus, the sink device 150 obtainsboth video signal total latency (100 ms) and audio signal total delaytime before correction (90 ns), and controls the audio variable delayunit 151 on the basis of the difference 10 ms as a correction value(AD). As a result, the audio signal total delay time is corrected to 100ms.

When audio and video signals of the DVD medium 101 are synchronized inthe source device 100, the video signal reproduced in the sink device140 and the audio signal reproduced in the sink device 150 are bothdelayed by 100 ms. Thus, audio and video signals are finallysynchronized and Lip-sync reproduction is realized.

3. Summary

In the embodiment, as described herein, each device on the videotransmission path accumulates sequentially the delay time (latency) ofown video signal, and transmits the result to the video transmissionpath according to the request from the source device. The source deviceacquires the video signal total delay time on the transmission pathextending from the source device to the sink device, and transmits theparameter of video signal total delay time to the audio signaltransmission path together with the audio signal and audio signal delaytime. Each device on the audio transmission path sequentiallyaccumulates the delay time of own audio signal, and transmits the resultto the audio transmission path. As a result, the final device of theaudio transmission path can know the video signal total delay time andaudio signal total delay time. Accordingly, providing the extra delay ofan audio signal to decrease the difference between the total delay timescan correct deviation in Lip-sync. This advantage can be obtained evenif the end points of the audio and video paths are different, and it isnot affected by topology. Instead of time stamps, constant parameterinformation of latency and delay time of each device on each path can beutilized, and thus number of data transmission can be reduced.

The functions described above operate as entire system for composing thenetwork by adding specified functions to the interface for connectingthe individual devices. Therefore, when the devices have specifiedfunctions, the device can be replaced with others in the network, andthe same action and effect can be obtained even if the networkconnection configuration (topology) is changed. In the followingembodiments, several applications in various network connections(topology) are explained.

Embodiment 2

This embodiment explains an application example of the invention insystem configuration realized by direct connection of a source deviceand a sink device. The source device is a DVD player, and the sinkdevice is a digital television receiver (digital TV).

FIGS. 15A, 15B and 15C show a system configuration in the embodiment,respectively. A video signal and an audio signal are transmitted fromthe DVD player 100 as a source device to the digital TV 140 as a sinkdevice, and both video and audio signals are reproduced in the digitalTV 140.

The source device 100 is means for reproducing a DVD medium 101 andoutputs both video signal 200 and audio signal 300 in conformity withHDMI. The sink device 140 incorporates a video signal processing unit142 and an audio decoder 152, and outputs video and audio through a LCD143 and a speaker 153. The speaker 153 includes an audio signalamplifier. Video signal latency Lv of the video signal processing unit142 is supposed to be 80 ms, and audio signal latency La of the decoder152 is supposed to be 10 ms.

FIG. 15A shows an example in which a source device 100 is a conventionaldevice. Therefore, the source device 100 cannot acquire the video signaltotal latency TLv from the sink device 140, or cannot add it to theaudio signal. As a matter of fact, information of the audio signalcumulative delay time sDa cannot be added to the audio signal. On theother hand, the sink device 140 recognizes that such additionalinformation is not added to the audio signal of the source device 100.Since there is no additional information, the sink device 140 correctsLip-sync by itself. Specifically, on the basis of the video signallatency Lv (80 ms) in the sink device 140 and the latency La (10 ms) ofthe decoder 152, an extra delay of 70 ms (=80−10) is produced forcorrecting the time difference in the audio delay unit 151. The extradelay allows the total value of delay in both video signal and audiosignal to be 80 ms, thus achieving Lip-sync correction.

FIGS. 15B and 15C show examples with a source device 100 to which theconcept of the present invention is applied. The source device 100 canacquire the audio signal cumulative delay time in addition to the videosignal total latency TLv through EDID line, and further add these piecesof information to the audio signal. FIG. 15B shows the source device 100having the function of the controller 420 in FIG. 1 with an extra delayadded to the source device 100. FIG. 15C shows an example in which thefunction of the controller 420 in FIG. 1 is dispersed into the sourcedevice 100 and the sink device 140 and the extra delay is also dispersedinto the source device 100 and the sink device 140.

In FIG. 15B, the source device 100 acquires the audio signal latency La(10 ms) in addition to the video signal total latency TLv (80 ms) fromthe sink device 140 through EDID line. Hence, the source device 100 cancalculate the extra delay (70 ms) from the acquired information andproduce the audio signal which is delayed by the extra delay (70 ms).Because of the extra delay, the source device 100 transmits the audiosignal cumulative delay time sDa as 70 ms, as additional information ofthe audio signal 300. The sink device 140 can recognize that the videosignal total latency is 80 ms, and that the audio signal cumulativedelay time is 70 ms. Further, the sink device 140 can recognize that thelatency of the decoder 152 is 10 ms and hence the audio signal totallatency is 80 ms, and that further delay process is not necessary. Thus,the Lip-sync can be corrected in the system.

FIG. 15C is similar to FIG. 15B. However, the extra delay in the sourcedevice 100 is limited to 40 ms. The source device 100 delays the audiosignal by the extra delay of 40 ms and outputs it. Due to the extradelay, the source device 100 adds the audio signal cumulative delay timesDa as 40 ms to the audio signal 300, and transmits it. The sink device140 can recognize the video signal total latency TLv (80 ms), the audiosignal cumulative delay time sDa (40 ms) and the latency (10 ms) of thedecoder 152. Hence, the sink device 140 can recognize that thedifference in delay time of a video signal and an audio signal is 30 ms.Accordingly, the sink device 140 processes the extra delay of 30 ms inthe audio delay unit 151. As a result, in both video signal and audiosignal, the total delay time is corrected to 80 ms, so that the Lip-synccan be corrected in the system.

Embodiment 3

This embodiment is different from embodiment 2 in that a repeater isinserted in the audio transmission path.

FIGS. 16A, 16B and 16C show a system configuration of the presentembodiment, respectively. A sink device 140 as a digital TV is connectedto a source device 100 as a DVD player via the repeater 110 as amulti-channel amplifier.

The source device 100 outputs a video signal 200 and an audio signal 300reproduced from the DVD medium 101 in conformity with HDMI.

The repeater 110 has an audio signal processing function, and is usedfor outputting higher quality audio sound than built-in amplifier orspeaker of the sink device 140, or achieving multi-channel. The repeater110 has a re-encoder 112 and transmits the output to the sink device140. In the repeater 110, the video signal is passed through withoutdelay, but the audio signal is delayed by latency La of 10 ms with there-encoder 112.

The sink device 140 is a digital TV, incorporates a video signalprocessing unit 142 and an audio decoder 152, and presents video andaudio through a LCD 143 and a speaker 153. In the sink device 140, thelatency Lv of the video signal processing unit 142 is 80 ms, and thelatency La of the decoder 152 is 10 ms.

FIG. 16A shows an example in which a source device 100 is a conventionaldevice. Therefore, the source device 100 can neither recognize the videosignal total latency TL, nor add the information to the audio signal.The audio signal cumulative delay time sDa cannot be added to the audiosignal.

The repeater 110 does not receive information of the audio signalcumulative delay time sDa added to the audio signal 300 through EDIDline, and hence recognizes that additional information is absent.Accordingly, the repeater 110 operates with the same functions as thesource device of the present invention. That is, the repeater 110receives video signal latency Lv (80 ms) from the sink device 140through EDID line, and recognizes that the video signal total latencyTLv is 80 ms. The repeater 110 transmits information of the totallatency TLv (80 ms) to the sink device 140 as additional information ofthe audio signal. Further, the repeater 110 multiplexes the audio signallatency on the audio signal as cumulative delay time sDa (10 ms), andtransmits the result to the sink device 140.

Output of re-encoder 112 in the repeater 110 is supplied to the speaker113 by way of the audio variable delay unit 111, a multi-channelamplifier (not shown) and others in the repeater 110. At this time, therepeater 110 controls the delay time of the audio variable delay unit111 on the basis of difference (70 ms) between the video signal totallatency TLv (80 ms) and the audio signal cumulative delay time sDa (10ms). As a result, Lip-sync deviation between the audio output from thespeaker 113 and the video output reproduced from the LCD 143 iscorrected. Thus, Lip-sync can be corrected for the audio signal from therepeater 110 added for the purpose of higher sound quality.

As for the audio from the speaker of the sink device 140 usablesubsidiarily, the Lip-sync can be corrected as follows. The sink device140 calculates 60 ms as time difference between video and audio outputs,on the basis of the information from the repeater 110, from the totallatency TLv (80 ms), the audio signal cumulative delay time sDa (10 ms),and the latency (10 ms) of the own decoder 152. On the basis of thisinformation of 60 ms, the sink device 140 controls the audio variabledelay unit 151 to correct each total latency commonly to 80 ms. Thus,the audio from the sink device 140 can be also corrected in Lip-sync.

FIGS. 16B and 16C show examples with a source device 100 to which theconcept of the invention is applied. The source device 100 acquiresaudio signal cumulative delay time sDa in addition to video signal totallatency TLv through EDID line, and can further add these pieces ofinformation to the audio signal.

FIG. 16B shows the source device 100 which acquires the total latency ofeach of a video signal and an audio signal from the repeater 110 and thesink device 140 through the EDID line. The video signal total latencyTLv is acquired by issuing the TLv send command to the downstreamdevices, as mentioned in Embodiment 1. The source device 100 acquiresthe video signal total latency 80 ms.

The source device 100 issues a command for acquiring the audio signaltotal latency to the downstream devices. Each downstream device whenreceiving this command, sends it to the upstream devices in the same wayas the case of TLv send command, while adding the own latency La to theaudio signal cumulative delay time sent from the downstream. As aresult, the source device 100 recognizes the audio signal total latencyis 20 ms (=10 ms+10 ms).

The video signal total latency is 80 ms and the audio signal totallatency is 20 ms, and hence the source device 100 sets 60 ms (=80 ms−20ms) as extra delay, and adds the extra delay to the audio signal. Thesource device 100 transmits information of the video signal totallatency TLv (80 ms) and the audio signal cumulative delay time sDa (60ms), as additional information of audio signal.

The repeater 110 receives these pieces of information, and controls theaudio variable delay unit 111 so as to correct the difference of 20 msof video signal total latency TLv (80 ms) and audio signal cumulativedelay time sDa (60 ms). As a result, Lip-sync deviation of the audiooutput from the speaker 113 and video reproduced from the LCD 143 iscorrected. Thus, Lip-sync can be corrected for the audio signal from therepeater 110 which is added for the purpose of higher sound quality.

As for the audio from the speaker of the sink device 140 usablesubsidiarily, the Lip-sync can be corrected as follows. The sink device140 recognizes that the video signal total latency TLv is 80 ms and theaudio signal cumulative delay time sDa is 70 ms, on the basis of theinformation from the repeater 110, and calculates 10 ms as delay timedifference of video signal and audio signal. Since this time differenceof 10 ms is equal to the latency 10 ms of the sink device 140, and thesink device 140 outputs the audio signal without adding the extra delay.Thus, the audio from the sink device 140 can be also corrected inLip-sync.

FIG. 16C is similar to FIG. 16B. However, the extra delay in the sourcedevice 100 is limited to 40 ms. Accordingly, the source device 100 addsthe extra delay of 40 ms to the audio signal and outputs it. Due to thisextra delay, the source device 100 transmits information of the audiosignal cumulative delay time sDa (40 ms) in addition to the video signaltotal latency TLv (80 ms) as additional information of the audio signal300. The repeater 110 and the sink device 140 receive this additionalinformation. Then, the repeater 110 controls its operation so that theextra delay of the audio variable delay unit 111 may be 30 ms, and thesink device 140 controls its operation so that the extra delay of theaudio variable delay unit 151 may be 20 ms, respectively. Thus, in bothmain and sub audio outputs, the audio total delay time is corrected to80 ms, and the Lip-sync can be corrected in the entire system.

Embodiment 4

This embodiment explains an example in which the concept of theinvention is applied to a configuration in which a source device and asink device are connected each other and an audio transmission path isseparated to plural paths with an amplifier connected to one of theseparated paths. A source device (DVD player) 100, a sink device(multi-channel amplifier) 150, and a sink device (digital TV) 140 areconnected through HDMI.

FIGS. 17A, 17B and 17C show a system configuration of the embodiment,respectively. The sink device 150 is a multi-channel amplifier, andincludes a decoder 152, an audio variable delay unit 151, and aplurality of amplifiers and speakers. The source device 100 can acquirethe video signal total latency TLv and the audio signal cumulative delaytime sDa, and add them to the audio signal.

FIG. 17A shows an example in which the source device 100 does not addthe extra delay to the audio signal. The source device 100 transmitsinformation of the video signal total latency TLv (80 ms) to the sinkdevice 140 as additional information of the audio signal 300 throughEDID line. At the same time, the source device 100 adds the audio signalcumulative delay time sDa (0 ms) to the audio signal 300, and transmitsthe result to the sink device 140 and the sink device 150.

The sink device 140 calculates the delay time difference 70 ms from thereceived video signal total latency TLv (80 ms), audio signal cumulativedelay time sDa (0 ms), and audio signal latency La (10 ms) of the sinkdevice 140, and controls the audio variable delay unit 251 to add anextra delay of 70 ms.

Similarly, the sink device 150 calculates the time difference 60 ms fromthe audio signal latency La (20 ms) of the sink device 150, and controlsthe audio variable delay unit 151 to add an extra delay of 60 ms. Thus,the Lip-sync of LCD 143, speaker 153, and speaker 253 can be corrected.

FIGS. 17B and 17C show an example in which the extra delay of an audiosignal is added in the source device 100, respectively.

In FIG. 17B, the source device 100 preliminarily knows the audio signallatency La of the sink device 140 and the sink device 150 through EDIDline. According to this information, the source device 100 sets an extradelay of 60 ms in the audio variable delay unit 102 on the basis of thepath to the sink device 150 which provides the maximum latency La.Information of the total latency TLv (80 ms) and the audio signalcumulative delay time sDa (60 ms) is multiplexed on the audio signal300, and the multiplexed information on the audio signal 300 istransmitted to the sink device 140 and the sink device 150. In thiscase, no extra delay is needed at the sink device 150. The sink device140 calculates the output time difference 10 ms between audio and videooutputs from the received video signal total latency TLv (80 ms), theaudio signal cumulative delay time sDa (60 ms), and the audio signallatency La (10 ms) of the sink device 140, and controls the audiovariable delay unit 251 to provide the extra delay of 10 ms. Thus, theLip-sync of LCD 143, speaker 153, and speaker 253 can be corrected.

In FIG. 17C, the source device 100 preliminarily knows the audio signallatency La of the sink device 140 and the sink device 150 through EDIDline. Accordingly, from this, on the basis of the audio transmissionpath for providing the maximum latency La, the source device 100 setsthe extra delay of 30 ms in the audio variable delay unit 102. Thesource device 100 multiplexes the video signal total latency TLv (80 ms)and the audio signal cumulative delay time sDa (30 ms) on the audiosignal 300, and transmits the multiplexing result to the sink device 140and the sink device 150. In this case, no extra delay is needed at thesink device 140. The sink device 150 calculates the time difference 30ms by using the audio signal latency La (20 ms) of the sink device 150,from the received video signal total latency TLv (80 ms) and the audiosignal cumulative delay time sDa (30 ms), and controls the audiovariable delay unit 151 to provide an extra delay of 30 ms. Thus, theLip-sync of LCD 143, speaker 153, and speaker 253 can be corrected.

Embodiment 5

This embodiment explains an example in which the concept of theinvention is applied to a configuration in which a source device and asink device are connected each other and an audio transmission path isbranched with an amplifier connected to a branch.

FIGS. 18A, 18B and 18C show a system configuration of the embodiment,respectively. In the embodiment, a source device (DVD player) 100 and asink device (digital TV) 140 are connected through HDMI. Further, thesource device (DVD player) 100 and a sink device (multi-channelamplifier) 150 are connected with S/PDIF interface. S/PDIF interface isan interface for transmitting digital audio or the like by using acoaxial or optical cable specified in IEC60958. Since S/PDIF interfaceis for a one-way transmission from the source device to the sink device,S/PDIF interface does not have an up-line from the sink device to thesource device.

FIG. 18A shows an example in which the source device 100 does not add anextra delay to an audio signal. The source device 100 transmitsinformation of the video signal total latency TLv (80 ms) to the sinkdevice 140 as additional information of the audio signal 300 throughEDID line, and multiplexes the audio signal cumulative delay time sDa (0ms) to the audio signal 300 and transmits the multiplexed signal to thesink device 140 and the sink device 150.

The sink device 140 calculates the delay time difference 70 ms betweenvideo and audio signals from the received video signal total latency TLv(80 ms) and audio signal cumulative delay time sDa (0 ms), and the audiosignal latency La (10 ms) of the sink device 140, and controls the audiovariable delay unit 251 to provide an extra delay of 70 ms.

Similarly, the sink device 150 calculates the delay time difference 60ms from the received video signal total latency TLv (80 ms) and audiosignal cumulative delay time sDa (0 ms), and the audio signal latency La(20 ms) of the sink device 150, and controls the audio variable delayunit 151 to add an extra delay of 60 ms. Thus, the Lip-sync of LCD 143,speaker 153, and speaker 253 can be corrected.

FIGS. 18B and 18C show an example in which an extra delay of an audiosignal is added by the source device 100, respectively.

In FIG. 18B, the sink device 150 is connected to the source device 100through S/PDIF interface of one-way transmission, and thus the sourcedevice 100 cannot preliminarily know the audio signal latency La of thesink device 150. In the source device 100, however, the extra delay 70ms is set preliminarily in the audio variable delay unit 102. The sourcedevice 100 multiplexes the total latency TLv (80 ms) and the audiosignal cumulative delay time sDa=70 ms on the audio signal 300, andtransmits the multiplexing result to the sink devices 140 and 150.

In this case, no extra delay is needed at the sink device 140. The sinkdevice 150 calculates the delay time difference −10 ms from the receivedvideo signal total latency TLv (80 ms), the audio signal cumulativedelay time sDa (70 ms), and the audio signal latency La (20 ms) of thesink device 150. To correct the delay time difference −10 ms, a videosignal must be delayed. However it is impossible to do so in theembodiment, and therefore the negative delay time difference is ignored.As a result, the audio signal is delayed by 10 ms from the video signal.This phenomenon occurs often in nature due to difference in velocitybetween light and sound, and human sense does not feel strange with atime difference of 100 ms or less. Hence there is no particular problemif such negative delay time difference is ignored.

In FIG. 18C, the source device 100 preliminarily knows the audio signallatency La of the sink device 140 through EDID line. According to this,the source device 100 sets an extra delay of 30 ms in the audio variabledelay unit 102. The source device 100 multiplexes the video signal totallatency TLv (80 ms) and the audio signal cumulative delay time sDa (30ms) on the audio signal 300, and transmits the multiplexing result tothe sink devices 140 and 150. In this case, no extra delay is needed atthe sink device 140. The sink device 150 calculates the delay timedifference 30 ms, from the received video signal total latency TLv (80ms) and the audio signal cumulative delay time sDa (30 ms), and theaudio signal latency La (20 ms) of the sink device 150, and controls theaudio variable delay unit 151 to provide an extra delay of 30 ms. Thus,the Lip-sync of LCD 143, speaker 153, and speaker 253 can be corrected.

Embodiment 6

This embodiment explains an example in which the concept of theinvention is applied to a configuration in which a source device and asink device are connected each other and an audio transmission pathextends from one sink device to the other sink device.

FIGS. 19A and 19B show a system configuration of the embodiment,respectively. A source device (DVD player) 100 is connected to a sinkdevice (digital TV) 140, and further an audio signal is connected fromthe sink device (digital TV) 140 to a sink device (multi-channelamplifier) 150 through HDMI.

In FIG. 19A, the source device 100 is a conventional device. Therefore,the source device 100 can neither recognize the video signal totallatency TLv nor add it to an audio signal, and cannot add the audiosignal cumulative delay time.

On the other hand, the sink device 140 recognizes that the audio signalproduced from the source device 100 does not contain such additionalinformation. Since no additional information is present, the sink device140 corrects the Lip-sync by it self. Specifically, the sink device 140preliminarily acquires the audio signal latency La (20 ms) of the sinkdevice 150 through EDID line, adds it to the own latency La (10 ms) tocalculate the latency La as 30 ms. The sink device 140 generates anextra delay of 50 ms for correcting the time difference in the audiovariable delay unit 251 from the video latency Lv (80 ms) and thecalculated latency La (30 ms). Accordingly, the sink device 140 obtainsthe total latency 80 ms for both a video signal and an audio signal,thus achieving Lip-sync correction.

FIG. 19B shows an example in which the concept of the invention isapplied to the source device 100. The source device 100 can recognizethe latency of the sink devices 140 and 150, and thus can correct thedelay more flexibly. An extra delay of 50 ms is provided in the sourcedevice 100. This allows the total latency of both video signal and audiosignal to be set to 80 ms finally in the sink device 150, thus achievingLip-sync correction.

Embodiment 7

This embodiment explains an example in which the concept of theinvention is applied to a configuration in which a repeater (videoprocessor) 120 inserted between the source device 100 and the sinkdevice 140 is added to the system configuration shown in FIG. 17. Alldevices are connected via HDMI interface. In the embodiment, therepeater 120 is a video processor for processing the video signal in apredetermined way to enhance the picture quality.

FIGS. 20A, 20B and 20C show a system configuration of the embodiment,respectively. The repeater 120 has the video signal latency Lv of 50 ms.The source device 100 can recognize the video signal total latency TLv(130 ms) through the repeater 120, and add the information to an audiosignal, and further can add audio signal cumulative delay time.

FIG. 20A shows an example in which the source device 100 does notprovide an extra delay of an audio signal. The source device 100transmits information of the total latency TLv (130 ms) to the audiosignal 300, multiplexes the cumulative delay time sDa (0 ms) for theaudio signal latency on the audio signal, and transmits the multiplexingresult. On the basis of the received information, the repeater adds anextra delay of 40 ms to the audio signal.

The sink device 140 calculates the delay time difference 80 ms from thereceived video signal total latency TLv (130 ms) and audio signalcumulative delay time sDa (40 ms), and the audio signal latency La (10ms) of the sink device 140, and controls the audio variable delay unit251 to provide an extra delay.

Similarly, the sink device 150 calculates the delay time difference 110ms from the received video signal total latency TLv (130 ms) and audiosignal cumulative delay time sDa (40 ms), and the audio signal latencyLa (20 ms) of the sink device 150, and controls the audio variable delayunit 151 to provide an extra delay of 110 ms. Thus, the Lip-sync of LCD143, speaker 153, and speaker 253 can be corrected.

FIGS. 20B and 20C show an example in which an extra delay of an audiosignal is added in the source device 100, respectively. In FIG. 20B, thesource device 100 sets an extra delay of 110 ms in the audio variabledelay unit 102. In this case, no extra delay is needed at the sinkdevice 150.

The sink device 140 calculates the delay time difference 10 ms from thereceived video signal total latency TLv (130 ms) and audio signalcumulative delay time sDa (110 ms), and the audio signal latency La (10ms) of the sink device 140, and controls the audio variable delay unit251 to provide an extra delay of 10 ms. Thus, the Lip-sync of LCD 143,speaker 153, and speaker 253 can be corrected.

In FIG. 20C, the source device 100 sets an extra delay 80 ms in theaudio variable delay unit 102. The source device 100 multiplexes thevideo signal total latency TLv (130 ms) and the audio signal cumulativedelay time sDa (80 ms) on the audio signal 300, and transmits themultiplexing result. In this case, no extra delay is needed at the sinkdevice 140.

The sink device 150 calculates the delay time difference 30 ms from thereceived video signal total latency TLv (130 ms) and audio signalcumulative delay time sDa (80 ms), and the audio signal latency La (20ms) of the sink device 150, and controls the audio variable delay unit151 to provide an extra delay of 30 ms. Thus, the Lip-sync of LCD 143,speaker 153, and speaker 253 can be corrected.

Embodiment 8

This embodiment explains an example in which the concept of theinvention is applied to a configuration in which a repeater (videoprocessor) 110 is inserted between a source device (DVD player) 100 anda sink device (digital TV) 140 and a sink device (multi-channelamplifier) 150 is connected to the source device via the repeater 110.The repeater 110 and sink device 150 are connected in S/PDIF interface,and other devices are connected in HDMI.

FIGS. 21A, 21B and 21C show a system configuration of the embodiment,respectively. In FIG. 21A, the source device 100 is a conventionaldevice which cannot recognize the video signal total latency (TLv).

The repeater 110 recognizes the video signal total latency TLv (80 ms)through EDID line. The repeater 110 sends the video signal total latencyTLv (80 ms) and the audio signal cumulative delay time sDa (20 ms) tothe downstream devices.

The sink device 140 calculates the delay time difference 50 ms from thevideo signal total latency TLv (80 ms) and audio signal cumulative delaytime sDa (20 ms) which are received from the repeater 110, and thelatency La (10 ms) of itself. The audio variable delay unit 251 iscontrolled on the basis of the calculated time difference 50 ms.

The sink device 150 receives the video signal total latency TLv (80 ms)and the audio signal cumulative delay time sDa (20 ms) through S/PDIF,calculates the delay time difference 40 ms from the received values andthe own latency La (40 ms), and controls the audio variable delay unit151 on the basis of the calculated value. Thus, all Lip-sync can becorrected.

FIGS. 21B and 21C show an example that an extra delay of an audio signalis provided in the source device 100, respectively. In FIG. 21B, sincethe repeater 110 and the sink device 150 are connected via S/PDIFinterface, the source device 100 cannot preliminarily know the audiosignal latency La of the sink device 150, but sets an extra delay of 50ms in the audio variable delay unit 102 preliminarily.

The source device 100 multiplexes the video signal total latency TLv (80ms) and the audio signal cumulative delay time sDa (50 ms) on the audiosignal, and transmits the multiplexing result to the repeater 110. Inthis case, no extra delay is needed at the sink device 140.

The sink device 150 calculates the delay time difference −10 ms from thereceived video signal total latency TLv (80 ms), the audio signalcumulative delay time sDa (50 ms), and the audio signal latency La (20ms) of the sink device 150. To correct the delay time difference −10 ms,it is necessary to delay the video signal, not the audio signal. Howeverit is impossible in this embodiment to do so, and hence the delay timedifference is ignored. The reason of ignoring such negative delay timedifference is as described before.

In FIG. 21C, the source device 100 preliminarily knows the audio signallatency La (50 ms) of the sink device 140 through EDID line. On thebasis of this value, the source device 100 sets an extra delay 10 ms inthe audio variable delay unit 102. As a result, no extra delay is neededat the sink device 140.

The sink device 150 calculates the cumulative delay time difference 30ms from the received video signal total latency TLv (80 ms) and audiosignal cumulative delay time sDa (30 ms), and the audio signal latencyLa (20 ms) of the sink device 150, and controls the audio variable delayunit 151 to provide an extra delay of 30 ms.

Thus, the Lip-sync of all LCD and speakers can be corrected.

Embodiment 9

Referring to FIG. 22, a method of superimposing additional informationon the audio signal is explained. FIG. 22 is an explanatory diagram ofan example of transmission of additional information by using the userbit (Ubit) specified in IEC60958.

Ubit is included in sample data of the audio signal, and one bit isadded to every PCM data of L and R channels. Ubit is a bit row, and isdefined in a block of 8 bits starting from a start bit. Ubit isspecified to use General user data format. Therefore, conforming toGeneral user data format, latency information and other additionalinformation are additionally defined. Details of General user dataformat are described in IEC60958, in section 3.6.2.4.1.

FIG. 23 shows the format of additional information, such as latencyinformation, additionally defined according to General user data format.The first IU1 shows to be latency information, and IU2 defines wordlength of information. IU3 is a copy of category code. By using eight IUunits from IU4 to IU11, information of six bytes including the followingitems is composed.

Audio Latency Valid: Validity bit of information of audio signalcumulative delay time sDa.

Audio Unit type: Unit of information of audio signal cumulative delaytime sDa.

Audio Latency: Information of audio signal cumulative delay time sDa,having word length of 16 bits.

Video Latency Valid: Validity bit of information of video signal totallatency TLv.

Video Unit type: Unit of information of video signal total latency TLv.

Video Latency: Information of video signal total latency TLv, havingword length of 16 bits.

Thus, transmission using Ubit is intended to keep generality wideregardless of category. Further, time codes of SMPTE standard can betransmitted.

By using the digital audio interface conforming to the IEC standard inthis manner, latency information necessary for the present invention canbe transmitted. That is, the digital audio interface conforming to theIEC standard can be applied in the S/PDIF interface in the foregoingembodiments.

In HDMI, the audio transmission format in conformity with IEC60958 isspecified. In other words, IEC60958 format is transmitted as beingincluded in HDMI format. That is, by using Ubit in the exactly same wayas IEC60958, all latency information used in the invention can betransmitted in the format of IEC60958 also in HDMI. Further by using theformat called IEC conformant specified by AMP protocol of IEC61883(1394), the latency information of the invention can be transmitted bythe 1394 interface exactly in the same manner.

The invention is specifically described herein by referring to specifiedembodiments, but may be changed or modified in various forms by thoseskilled in the art without departing from the spirit of the essentialcharacteristics thereof. Therefore, the invention is defined only by theappended claims rather than by the description preceding them. Thisapplication is related to Japanese Patent Application No. 2005-131963(filed on Apr. 28, 2005), the entire contents of which are incorporatedherein by reference.

INDUSTRIAL APPLICABILITY

The present invention is particularly useful in AV devices processingaudio and video signals, networks with interfaces connecting those AVdevices, and other apparatuses, AV appliances, and interfaces in a widerange.

1. A Lip-sync correction system for correcting difference inreproduction time between a video signal and an audio signal, wherein avideo signal and an audio signal are transmitted from a source device toa sink device through the respective transmission paths in conformitywith a specified interface, individually, the specified interfaceincludes a bidirectional interface, and the system comprises acontroller for acquiring delay information indicating the total delaytime of devices on the transmission paths conforming to thebidirectional interface, and correcting the difference in reproductiontime by using the acquired delay information.
 2. The Lip-sync correctionsystem according to claim 1, wherein the video signal is transmittedwith a bidirectional interface, and the audio signal is transmitted witha one-way interface.
 3. The Lip-sync correction system according toclaim 1, wherein the controller applies an extra delay to the device onthe transmission path of an audio signal in order to correct thedifference in reproduction time.
 4. The Lip-sync correction systemaccording to claim 3, wherein the controller provides an extra delay tothe device at the lowermost downstream on the transmission path of anaudio signal.
 5. The Lip-sync correction system according to claim 1,wherein the delay information is transmitted as additional informationof an audio signal.
 6. The Lip-sync correction system according to claim1, wherein the delay information is delay information about delay timein video signal processing and/or audio signal processing.
 7. TheLip-sync correction system according to claim 1, wherein thebidirectional interface is either one of HDMI (High DefinitionMultimedia Interface) and IEEE1394.
 8. A video reproducing apparatus forgenerating and outputting a video signal and an audio signal,comprising: a video interface unit that is connected to a video signaltransmission path and operable to transmit a video signal in conformitywith a bidirectional interface; an audio interface unit that isconnected to an audio signal transmission path and is operable totransmit an audio signal; and a controller operable to acquire delayinformation indicating total delay time of devices connected to thevideo signal transmission path, and transmitting the acquired delayinformation to a device connected to the audio signal transmission path.9. An audio output device for receiving an audio signal and outputtingaudio on the basis of the received audio signal, comprising: aninterface unit operable to receive delay information about a videosignal and delay information about an audio signal, together with theaudio signal; and a controller operable to adjust output time of anaudio signal on the basis of the received delay information.
 10. A videodisplay device for receiving a video signal and presenting video on thebasis of the received video signal, comprising: an interface unitoperable to receive a command for requesting transmission of delay time;and a controller operable to transmit the delay time of a video signalfor the video display device through the interface when receiving thecommand.
 11. A Lip-sync correcting method for correcting difference inreproduction time between a video signal and an audio signal,comprising: a) transmitting a video signal and an audio signal from asource device to a sink device through the respective transmission pathsin conformity with the respective specified interfaces, individually,the specified interfaces including a bidirectional interface; b)acquiring delay information indicating the total delay time of deviceson the transmission paths conforming to the bidirectional interface; andc) correcting the difference in reproduction time between a video signaland a audio signal by using the acquired delay information.